Iván Palomares Carrascosa writes about methods for interpreting the dense numerical vector representations, or embeddings, generated by large language models (LLMs). By using a combination of probing classifiers like logistic regression, UMAP dimensionality reduction for visualization, and SHAP values to identify influential latent dimensions, one can analyze the quality and semantic structure captured within LLM-generated embedding spaces.
- Probing classifiers help determine if embeddings are rich enough to distinguish between classes by testing them with simpler models.
- UMAP is used to project high-dimensional embeddings into 2D space for visual inspection of natural groupings.
- SHAP values can pinpoint which specific dimensions in an embedding most significantly influence a classifier's decisions.
- The article demonstrates using Scikit-LLM alongside local Ollama models to generate embeddings cost-effectively.
- Probing classifiers help determine if embeddings are rich enough to distinguish between classes by testing them with simpler models.
- UMAP is used to project high-dimensional embeddings into 2D space for visual inspection of natural groupings.
- SHAP values can pinpoint which specific dimensions in an embedding most significantly influence a classifier's decisions.
- The article demonstrates using Scikit-LLM alongside local Ollama models to generate embeddings cost-effectively.
This article provides a comprehensive guide on the basics of BERT (Bidirectional Encoder Representations from Transformers) models. It covers the architecture, use cases, and practical implementations, helping readers understand how to leverage BERT for natural language processing tasks.
A Github Gist containing a Python script for text classification using the TxTail API
